Pedestrian recognition apparatus and method

ABSTRACT

The present invention relates to a pedestrian recognition apparatus. A pedestrian recognition apparatus may comprises: an image receiving unit configured to sequentially receive a plurality of images from a camera; and a pedestrian determination unit configured to perform a pedestrian candidate object detection process of detecting a pedestrian candidate object from one or more objects in the images, a mobility determination process of determining whether the pedestrian candidate object is moving, using the plurality of images, and setting the pedestrian candidate object to a moving pedestrian candidate object, and a pedestrian possibility determination process of performing a predefined operation on the moving pedestrian candidate object, and setting the moving pedestrian candidate object to a pedestrian when a value calculated through the predefined operation is equal to or more than a threshold value.

TECHNICAL FIELD

The present disclosure relates to a pedestrian recognition technology,and more particularly, to a pedestrian recognition apparatus and methodcapable of recognizing an object around a vehicle and determiningwhether the recognized object is a pedestrian.

BACKGROUND ART

In general, an around view monitoring system refers to a system thatshows a situation in the range of 360 degrees around a vehicle throughfour cameras installed outside the vehicle. The around view monitoringsystem may display a situation around the vehicle on a monitor such thata driver feels as if the driver looked down from the sky, and allow thedriver to monitor a parking space or driving space around the vehicle.Therefore, the around view monitoring system may enable the driver toeasily park or drive the vehicle in a narrow space.

Korean Patent Registration No. 10-0499267 discloses a rear viewmonitoring system which is capable of displaying a rearward image and adistance to a rearward object such that a driver can check the rearwardobject when backing a vehicle or making a turn. Such a techniqueprovides a distance between the vehicle and the rearward object with therearward image, and allows the driver to conveniently and safely drivethe vehicle.

Korean Patent Publication No. 10-2013-0095525 discloses an around viewsystem and an around view providing system, which can provide an aroundview image and forward and rearward images of a vehicle, process theforward and rearward images such that more images can be displayed inthe traveling direction of the vehicle depending on a steering anglechange, and display the images on the screen. Such a technique allows adriver to have a better view in the traveling direction of the vehicle.

RELATED ART DOCUMENT Patent Document

(Patent Document 1) Korean Patent Registration No. 10-0499267

(Patent Document 2) Korean Patent Publication No. 10-2013-0095525

DISCLOSURE Technical Problem

Various embodiments are directed to a pedestrian recognition apparatuscapable of recognizing an object around a vehicle and determiningwhether the recognized object is a pedestrian.

Also, various embodiments are directed to a pedestrian recognitionapparatus capable of determining whether an object in an image receivedfrom a camera is a pedestrian, based on the mobility of the object, anddetermining a collision possibility with a vehicle.

Further, various embodiments are directed to a pedestrian recognitionapparatus capable of displaying a visual guidance on a pedestrianaccording to a collision possibility between the pedestrian and avehicle.

Technical Solution

In an embodiment, a pedestrian recognition apparatus may comprises: animage receiving unit configured to sequentially receive a plurality ofimages from a camera; and a pedestrian determination unit configured toperform a pedestrian candidate object detection process of detecting apedestrian candidate object from one or more objects in the images, amobility determination process of determining whether the pedestriancandidate object is moving, using the plurality of images, and settingthe pedestrian candidate object to a moving pedestrian candidate object,and a pedestrian possibility determination process of performing apredefined operation on the moving pedestrian candidate object, andsetting the moving pedestrian candidate object to a pedestrian when avalue calculated through the predefined operation is equal to or morethan a threshold value.

The pedestrian recognition apparatus further comprises a pedestriancollision possibility determination unit configured to determine thatthe pedestrian and the vehicle are likely to collide with each other,when the distance between the pedestrian and the vehicle decreases, andoverlay a visual guidance on the object.

The pedestrian determination unit cyclically performs the pedestriancandidate object detection process, the mobility determination processand the pedestrian possibility determination process, and determines thepossibility that the one or more objects are pedestrians.

The pedestrian determination unit further performs an object extractionprocess of extracting the one or more objects in the received imagebefore the pedestrian candidate object detection process.

The pedestrian determination unit detects vertical components of the oneor more objects in the pedestrian candidate object detection process,and determines a similarity between the detected vertical components anda pedestrian pattern.

The pedestrian determination unit determines the mobility of thepedestrian candidate object based on differences of the same pedestriancandidate object in images from the current time to a specific previoustime, during the mobility determination process.

During the pedestrian possibility determination process, the pedestriandetermination unit performs a HOG (Histogram of Oriented Gradient)operation on the moving pedestrian candidate object, performs an SVMweight operation on the HOG operation result, and sets the pedestriancandidate object to a pedestrian when the operation result is equal toor more than a preset threshold value.

The pedestrian determination unit estimates a region of the pedestriancandidate object, corresponding to the HOG operation region, based onthe near and far degree of the received image.

The pedestrian determination unit adjusts the size of the region of thepedestrian candidate object through a plurality of cells having avariable size.

The pedestrian recognition apparatus further comprises an autonomousemergency braking unit configured to check a collision possibility ofthe corresponding object with a vehicle, based on a distance between theobject and the vehicle, and control the vehicle to perform one or moreof velocity reduction and warning sound generation.

In other embodiment, a pedestrian recognition method may comprises:sequentially receiving a plurality of images from a camera; anddetecting a pedestrian candidate object from one or more objects in theimages, determining whether the pedestrian candidate object is moving,based on the plurality of images, setting the pedestrian candidateobject to a moving pedestrian candidate object, performing a predefinedoperation on the moving pedestrian candidate object, and setting themoving pedestrian candidate object to a pedestrian when a valuecalculated through the predefined operation is equal to or more than athreshold value.

The pedestrian recognition method further comprises determining that thevehicle and the pedestrian are likely to collide with each other, whenthe distance between the pedestrian and the vehicle decreases, andoverlaying a visual guidance on the object.

The pedestrian recognition method further comprises determining that thevehicle and the pedestrian are likely to collide with each other, whenthe distance between the pedestrian and the vehicle decreases, andcontrolling the vehicle to perform one or more of velocity reduction andwarning sound generation.

Advantageous Effects

In accordance with the embodiment of the present invention, thepedestrian recognition apparatus may recognize an object around avehicle, and determine whether the object is a pedestrian.

Furthermore, the pedestrian recognition apparatus may determine whetheran object in an image received from the camera is a pedestrian, based onthe mobility of the object, and determine a possibility of collisionwith the vehicle.

Furthermore, the pedestrian recognition apparatus may display a visualguidance on the pedestrian, depending on a collision possibility betweenthe pedestrian and the vehicle.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1A and 1B are plan views for describing a pedestrian recognitionapparatus in accordance with an embodiment of the present invention.

FIG. 2 is a block diagram illustrating the pedestrian recognitionapparatus of FIG. 1.

FIG. 3 is a block diagram illustrating a pedestrian determination unitof the pedestrian recognition apparatus of FIG. 1.

FIGS. 4A to 4C illustrate a process of performing a HOG operation on apedestrian candidate object.

FIG. 5 illustrates a process of expressing the directions of edges in acell and the values of the directions as histograms.

FIGS. 6A to 6G illustrate a process of determining a pedestrian byperforming an SVM weight operation on a descriptor vector.

FIGS. 7A and 7B illustrate a process of adjusting the size of apedestrian candidate object region by controlling the number of pixelsincluded in one cell.

FIG. 8 is a flowchart illustrating a pedestrian recognition processperformed in the pedestrian recognition apparatus of FIG. 1.

MODE FOR INVENTION

The description of the present invention is merely an example forstructural or functional explanation, and the scope of the presentinvention should not be construed as being limited by the embodimentsdescribed in the text. That is, the embodiments are to be construed asbeing variously embodied and having various forms, so that the scope ofthe present invention should be understood to include equivalentscapable of realizing technical ideas. Also, the purpose or effect of thepresent invention should not be construed as limiting the scope of thepresent invention, since it does not mean that a specific embodimentshould include all or only such effect.

Meanwhile, the meaning of the terms described in the present applicationshould be understood as follows.

The terms “first”, “second” and the like are intended to distinguish oneelement from another, and the scope of the right should not be limitedby these terms. For example, the first component may be referred to as asecond component, and similarly, the second component may also bereferred to as a first component.

It is to be understood that when an element is referred to as being“connected” to another element, it may be directly connected to theother element, but there may be other elements in between. On the otherhand, when an element is referred to as being “directly connected” toanother element, it should be understood that there are no otherelements in between. On the other hand, other expressions that describethe relationship between the components, such as “between” and “between”or “neighboring to” and “directly adjacent to” should be interpreted aswell.

The use of the singular should be understood to include pluralrepresentations unless the context clearly dictates otherwise, and theterms “comprise” or “having”, etc. are intended to include the features,numbers, steps, operations, components, It is to be understood that thecombinations are intended to specify the presence or absence of one ormore other features, integers, steps, operations, components, parts, orcombinations thereof.

In each step, the identification code (e.g., a, b, c, etc.) is used forconvenience of explanation, the identification code does not describethe order of the steps, Unless otherwise stated, it may occurdifferently from the stated order. That is, each step may occur in thesame order as described, may be performed substantially concurrently, ormay be performed in reverse order.

All terms used herein have the same meaning as commonly understood byone of ordinary skill in the art to which this invention belongs, unlessotherwise defined. Commonly used predefined terms should be interpretedto be consistent with the meanings in the context of the related art andcannot be interpreted as having ideal or overly formal meaning unlessexplicitly defined in the present application.

FIGS. 1A and 1 b are plan views for describing a pedestrian recognitionapparatus in accordance with an embodiment of the present invention.

Referring to FIG. 1A, the pedestrian recognition apparatus 100 may beinstalled in a vehicle 10, and determine whether an object around avehicle 10 is a pedestrian, based an image acquired through a camera 20.The camera 20 may generate one or more images by filming a situationaround the vehicle at one or more times. In an embodiment, the camera 20may be installed at the front of the vehicle 10, and film an environmentin front of the vehicle 10. In another embodiment, the camera 20 may beinstalled at the front, rear and both sides of the vehicle 10, and filmsurrounding environments at the front, rear and both sides of thevehicle 10.

When the object around the vehicle 10 corresponds to a pedestrian, thepedestrian recognition apparatus 100 may generate a visual guidancedepending on a collision possibility of the vehicle 10 with thepedestrian. The visual guidance corresponds to a guide which is visuallyprovided to a driver. The pedestrian recognition apparatus 100 maycontrol the vehicle to perform one or more of velocity reduction andwarning sound generation, depending on the collision possibility of thevehicle 10 with the pedestrian. Hereafter, the pedestrian recognitionapparatus 100 will be described in detail with reference to FIG. 2.

FIG. 2 is a block diagram illustrating the pedestrian recognitionapparatus of FIGS. 1A and 1B, and FIG. 3 is a block diagram illustratinga pedestrian determination unit of the pedestrian recognition apparatusof FIGS. 1A and 1B.

Referring to FIGS. 2 and 3, the pedestrian recognition apparatus 100includes an image receiving unit 210, a pedestrian determination unit220, a pedestrian collision possibility determination unit 230, anautonomous emergency braking unit 240 and a control unit 250.

The image receiving unit 210 receives an image from the camera 20. Theimage receiving unit 210 may generate an around view based on the imageacquired from the camera 20. The around view corresponds to a real-timeimage which is generated based on a plurality of images taken by thecameras 20 installed at the front, rear and both sides of the vehicle10, the real-time image allowing a driver to feel as if the driverlooked down at surrounding environments in the range of 360 degreesaround the vehicle 10.

In an embodiment, the image receiving unit 210 may generate atime-series around view by combining two or more around views based onthe images taken by the camera 20. The two or more around views may begenerated based on continuous-time images.

The pedestrian determination unit 220 includes an object extractionmodule 211, a pedestrian candidate detection module 212, a mobilitydetermination module 213 and a pedestrian possibility determinationmodule 214.

The object extraction module 211 extracts one or more objects from animage received from the camera 20 before a pedestrian candidatedetection process. The object may include both a dynamic object andstatic object. The dynamic object may include a pedestrian and animal,and the static object may include a tree and sign. The object extractionmodule 211 may detect edges (or boundary lines) in the image, andextract one or more objects each having an area equal to or more than apredetermined size distinguished by the edges. For example, the objectextraction module 211 extract an area equal to or more than apredetermined size distinguished by the edges as an object, and ignoreareas less than the predetermined size.

In an embodiment, the object extraction module 211 may extract one ormore objects based on a color difference between the background and theobject in the image. The object extraction module 211 may calculatepixel values of the image, group areas having similar color values, andextract one group as one object. The pixels of the object may be groupedinto one group based on the characteristic that the pixels have similarcolor values.

In another embodiment, the object extraction module 211 may detect aboundary line in an image and extract an object, using an edge detectionalgorithm such as the Canny edge detection algorithm, the line edgedetection algorithm or the Laplacian edge detection algorithm. Forexample, the object extraction module 211 may extract an object bygrouping areas distinguished from the background based on the detectedboundary line.

The pedestrian candidate detection module 212 detects a pedestriancandidate object from one or more objects extracted by the objectextraction module 211 through simple feature analysis. The pedestriancandidate object may include one or more objects which are likely tocorrespond to a pedestrian. The pedestrian candidate detection module212 may extract a feature corresponding to a specific feature of apedestrian (detection target object) from the object and compare theextracted feature to the specific feature of the pedestrian, therebypreviously removing an object unrelated to the pedestrian (detectiontarget object). Through this operation, the pedestrian recognitionapparatus in accordance with the present embodiment can reduce theoperation amount and improve the processing speed.

For example, the pedestrian candidate detection module 212 may detect avertical component for the extracted one or more objects, determine asimilarity between the detected vertical component and the pedestrianpattern, and detect a pedestrian candidate object. In an embodiment, thepedestrian candidate detection module 212 may analyze edges of theextracted object, and detect vertical components of the edges. When thevertical components of the edges of the extracted object are similar toa predefined pedestrian pattern, the pedestrian candidate detectionmodule 212 may primarily verify that the extracted object corresponds toa pedestrian candidate object. The predefined pedestrian pattern mayinclude an upper part and a lower part, which are defined based on thehorizontal line at a point from which the vertical component divergesinto two parts, the upper part having a length ranging from 60 to 140%of that of the lower part, and the lower part of the vertical componentdiverges into two parts in the longitudinal direction from thehorizontal line. For example, the pedestrian candidate detection module212 may detect vertical edges (or edges having a slope within apredetermined angle with the vertical direction) among the edges for theobject, compare a shape formed by the detected edges to the verticalreference shape of a person, stored in a predetermined table, and detectthe corresponding object as a pedestrian candidate object when thesimilarity between both shapes is equal to or more than a thresholdvalue.

The pedestrian candidate detection module 212 may analyze an internalregion of the edges for the primarily verified pedestrian candidateobject, and finally verify that the primarily verified pedestriancandidate object is a pedestrian candidate object, when the internalregion is similar to the pedestrian pattern. For example, the pedestrianpattern may correspond to a pattern having an empty region formed byedges in the lower part of the vertical component.

The mobility determination module 213 compares a previous image to acurrent image, and determines whether the pedestrian candidate object ismoving. The mobility determination module 213 may determine whether thepedestrian candidate object is moving, based on differences of the samepedestrian candidate object in images from the current time to aspecific previous time. At this time, the mobility determination module213 may determine the identity of the same pedestrian candidate object,based on the predefined pattern of the pedestrian candidate object.

In an embodiment, the mobility determination module 213 may extractmotion vectors for the same pedestrian candidate object from the imagesfrom the current time to a specific previous time, and detect one ormore of forward and backward movement and left and right movement. Themotion vector may indicate the direction and magnitude of a motion. Theforward and backward movement and the left and right movement may bedetected according to a change between a motion vector of the pedestriancandidate object in an image frame at a first time and a motion vectorof the pedestrian candidate object in an image frame at a second timecorresponding to a point of time before the first time.

In another embodiment, the mobility determination module 213 maydetermine whether the pedestrian candidate object is moving, using adifference between an image at a previous time and an image at thecurrent time.

For example, the mobility determination module 213 divides the imageframe of the current time (for example, k-th image frame) and the imageframe of the previous time (for example, (k−1)th image frame) into apredetermined size of blocks. The block may include m×n pixels where mand n are integers. The mobility determination module 213 may calculatedifferences in pixel value between a specific block selected from thek-th image frame and the respective blocks of the (k−1)th image frame,and determine whether the pedestrian candidate object is moving, basedon the pixel value differences.

For example, when a pedestrian candidate object is included in thespecific block (or blocks) of the k-th image frame, the mobilitydetermination module 213 compares the corresponding block of the k-thimage frame to a block located at a predetermined position in the(k−1)th image frame, and calculates differences in pixel value betweenthe corresponding pixels of the blocks. The mobility determinationmodule 213 calculates the sum of absolute difference (SAD) by summing upthe absolute values of the pixel value differences between thecorresponding pixels of the blocks. The SAD may be calculated throughEquation 1 below.

$\begin{matrix}{{S\; A\; D} = {\sum\limits_{i = 1}^{N}{\sum\limits_{j = 1}^{M}{\left( {{I_{ij}(k)} - {I_{ij}\left( {k - 1} \right)}} \right)}}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

In Equation 1, I_(ij) (k) represents the pixel value of an i-th row anda j-th column of a block in a k-th image frame, and I_(ij) (k−1)represents the pixel value of an i-th row and a j-th column of a blockin a (k−1)th image frame.

After calculating the SAD between the blocks corresponding to each otherat first, the mobility determination module 213 calculates an SAD whilechanging the positions of the specific block of the k-th image frame(block or blocks including the moving object candidate) and the block ofthe (k−1)th image frame corresponding to the specific block. Forexample, the mobility determination module 213 may calculate an SADbetween blocks while changing the positions of the blocks in a spiraldirection in the (k−1)th image frame.

After calculating the SAD of each block, the mobility determinationmodule 213 detects the block having the smallest SAD in the (k−1)thimage frame. The mobility determination module 213 may calculate theSADs by comparing a specific block of the k-th image frame (block orblocks including the pedestrian candidate object) to the blocks of the(k−1)th image frame. As a result, the block (blocks) of the (k−1)thimage frame, having the smallest SAD, may correspond to the specificblock of the k-th image frame.

The mobility determination module 213 may set the pedestrian candidateobject to a moving pedestrian candidate object, based on whether thepositions of blocks corresponding to each other in the k-th image frameand the (k−1)th image frame were changed. The pedestrian possibilitydetermination module 214 determines the possibility that the pedestriancandidate object is a pedestrian. The pedestrian possibilitydetermination module 214 performs a HOG (Histogram of Oriented Gradient)operation on the moving pedestrian candidate object, and performs an SVM(Support Vector Machine) on the HOG operation result for the movingpedestrian candidate object. When the value calculated through the SVMweight operation performed on the HOG operation result is equal to ormore than a preset threshold value, the pedestrian possibilitydetermination module 214 may set the moving pedestrian candidate objectto a pedestrian. The HOG operation indicating the directions of edgesusing histograms may be used when the shape of an object is notsignificantly changed and has a simple internal pattern and the objectcan be identified by the contour line of the object. In theabove-described embodiment, the k-th image frame and the (k−1)th imageframe were used as the current image frame and the previous image frame.The technical idea of present invention is not limited thereto. That is,it is obvious to those skilled in the art that the k-th image frame andthe (k−10)th image frame can be used as the current image frame and theprevious image frame. Therefore, the detailed descriptions thereof areomitted herein, in order not to obscure subject matters of the presentinvention.

FIGS. 4A to 4C illustrate the process of performing a HOG operation onthe pedestrian candidate object.

The pedestrian possibility determination module 214 extracts a regionincluding the moving pedestrian candidate object from an image frame inorder to perform a HOG operation.

For convenience of description, the following descriptions are based onthe supposition that the pedestrian possibility determination module 214extracts a region having a size of 64 pixels×128 pixels (FIG. 4A). Thepedestrian possibility determination module 214 defines the extractedregion using a predetermined size of cells and a predetermined size ofblocks. For example, the cell may have a size of 8 pixels×8 pixels, andthe block may have a size of 2 cells×2 cells or 3 cells×3 cells. Thesizes of the cell and the block may be changed depending on anembodiment.

For convenience of description, the following descriptions are based onthe supposition that the cell has a size of 8 pixels×8 pixels and theblock has a size of 2 cells×2 cells. The pedestrian possibilitydetermination module 214 may divide the extracted region into 8 cells×16cells or a total of 128 cells (FIG. 4B). The cells may be arrangedadjacent to each other. The pedestrian possibility determination module214 may define blocks each of which has a cell line overlapping a cellline of another block in the vertical or horizontal direction, therebydefining 7 blocks×15 blocks or a total of 105 blocks in the extractedregion (FIG. 4C)

The pedestrian possibility determination module 214 calculates thedirections of edges in a cell by performing a HOG operation on a cellbasis. In an embodiment, the pedestrian possibility determination module214 may standardize the directions of the angles into a predefinednumber of angle bins. FIG. 5 illustrates illustrating a process ofexpressing the directions of edges in a cell and the values of thedirections as histograms. Referring to FIG. 5, the directions of edgeswithin a cell are standardized into eight angle bins, and the values ofthe respective angle bins are expressed as histograms.

The pedestrian possibility determination module 214 performsnormalization on each block by dividing the values of the directions ofthe edges calculated for each cell by the average value of a block towhich the corresponding cell belongs. The pedestrian possibilitydetermination module 214 can reduce an influence of illumination orcontrast on an operation result through the normalization.

After performing normalization on each block, the pedestrian possibilitydetermination module 214 calculates a descriptor vector by enumeratingthe normalized values for each block. For example, the pedestrianpossibility determination module 214 may calculate a descriptor vectorby enumerating the normalized values for the directions of the edges foreach block in a predefined order. The descriptor vector indicates a HOGfeature of the corresponding block.

The pedestrian possibility determination module 214 may perform an SVMweight operation on the calculated descriptor vector, and set thecorresponding moving pedestrian candidate object to a pedestrian when avalue obtained by performing the SVM weight operation is equal to ormore than a preset threshold value.

FIGS. 6A to 6G illustrate the process of determining a pedestrian byperforming an SVM weight operation on a descriptor vector.

FIG. 6A illustrates an average gradient image of images for a detectiontarget object (pedestrian). FIG. 6B illustrates positive SVM weightswhich are calculated based on the average gradient image of FIG. 6A, andFIG. 6C illustrates negative SVM weights which are calculated based onthe average gradient image of FIG. 6A. The process of calculating an SVMweight based on the gradient image is obvious to those skilled in theart. Thus, the detailed descriptions thereof are omitted herein, inorder not to obscure subject matters of the present invention.

FIG. 6D illustrates a region including the pedestrian candidate object,extracted from an image frame, and FIG. 6E illustrates descriptorvectors calculated by performing a HOG operation on FIG. 6D. In FIG. 6D,as the values for the directions get higher, the values are displayedmore brightly, based on the descriptor vectors calculated for therespective cells.

FIG. 6F illustrates a result obtained by applying the positive SVMweights (FIG. 6B) to the descriptor vectors (FIG. 6D) calculated for therespective cells, and FIG. 6G illustrates a result obtained by applyingthe negative SVM weights (FIG. 6C) to the descriptor vectors (FIG. 6D)calculated for the respective cells. FIG. 6F illustrates descriptorvectors that are the nearest to the detection target object (pedestrian)among the descriptor vectors, and FIG. 6G illustrates descriptor vectorsthat are the farthest from the detection target object (pedestrian)among the descriptor vectors.

The pedestrian possibility determination module 214 may set thecorresponding moving pedestrian candidate object to a pedestrian when avalue obtained by applying the positive SVM weight to the calculateddescriptor vector is equal to or more than a preset threshold value.

In an embodiment, the pedestrian possibility determination module 214may estimate a region of the pedestrian candidate object correspondingto the HOG operation region based on the near and far degree of theimage received from the camera 20. When the moving object candidateobject is away from the camera, the moving pedestrian candidate objectmay be estimated to have a larger size than when the moving objectcandidate object is close to the camera. The near and far degree of themoving pedestrian candidate object may be decided based on a depth value(Z-axis). The region of the pedestrian candidate object may be set to arectangle including the moving pedestrian candidate object.

In an embodiment, the pedestrian possibility determination module 214may adjust the size of the moving pedestrian candidate object bycontrolling the number of pixels included in one cell. FIGS. 7A and 7Billustrate the process of adjusting the size of the moving pedestriancandidate object region by controlling the number of pixels included inone cell.

FIG. 7A illustrates that 8 pixels×8 pixels are included in one cell, andFIG. 7B illustrates that 6 pixels×6 pixels are included in one cell.

A moving pedestrian candidate object far from the camera may beextracted as a relatively small region, and a moving pedestriancandidate object near to the camera may be extracted as a relativelylarge region. When the size of a cell is fixed to a predetermined sizeregardless of a distance (for example, 8 pixels×8 pixels), the region ofthe moving pedestrian candidate object may be defined as a small numberof blocks, which may make the calculation complicated.

The pedestrian possibility determination module 214 may adjust thenumber of pixels included in one cell, and maintain the same numbers ofcells and blocks in the region of the moving pedestrian candidate objectregardless of the size of the corresponding region. Therefore, thepedestrian possibility determination module 214 can detect a smallerpedestrian (pedestrian farther from the camera) through the sametemplate, while adjusting the size of the region of the movingpedestrian candidate object by controlling the number of pixels includedin one cell.

Referring back to FIG. 2, the pedestrian determination unit 220 maycyclically perform the pedestrian candidate detection process, themobility determination process and the pedestrian possibilitydetermination process, thereby determining the possibility that one ormore objects are pedestrians. The pedestrian determination unit 220 mayraise the possibility that one or more objects are pedestrians, throughthe pedestrian candidate detection module 212, the mobilitydetermination module 213 and the pedestrian possibility determinationmodule 214.

The pedestrian collision possibility determination unit 230 determines acollision possibility of the object set to a pedestrian, based on themobility of the object. When the corresponding object is likely tocollide with the vehicle, the pedestrian collision possibilitydetermination unit 230 may transparently overlay a visual guidance onthe object, such that the object with the visual guidance is displayedon the screen.

When the object set to a pedestrian is away from the vehicle 10, thepedestrian collision possibility determination unit 230 may determinethat the object is unlikely to collide with the vehicle. On the otherhand, when the object set to a pedestrian approaches the vehicle 10, thepedestrian collision possibility determination unit 230 may determinethat the vehicle 10 is likely to collide with the object set to thepedestrian, based on the distance between the vehicle 10 and the object.For example, the pedestrian collision possibility determination unit 230may compare a previous image frame and the current image frame, anddetermine that the object set to a pedestrian is approaching thevehicle, when the object grows bigger in the image frames. On the otherhand, the pedestrian collision possibility determination unit 230 maydetermine that the object set to a pedestrian is being away from thevehicle, when the object grows smaller in the image frame.

In an embodiment, when the object set to a pedestrian is likely tocollide with the vehicle, the pedestrian collision possibilitydetermination unit 230 may highlight the region of the object set to apedestrian in the image received from the camera 20 or display an arrow(or indicator having a different shape) to visually inform a driver of acollision risk with the pedestrian. For example, the pedestriancollision possibility determination unit 230 may highlight the region inred when the collision risk is high or in yellow when the collision riskis medium.

The autonomous emergency braking unit 240 may check the collisionpossibility of the corresponding object with the vehicle 10, based onthe distance between the object and the vehicle 10, and control thevehicle 10 to perform one or more of velocity reduction and warningsound generation. The autonomous emergency braking unit 240 may decidethe distance between the corresponding object and the vehicle 10, basedon the current velocity of the vehicle 10 and the information indicatingwhether the object is moving. For example, the autonomous emergencybraking unit 240 may decide the distance between the object and thevehicle 10 based on the distance depending on the size of the region,and estimate the distance between the object and the vehicle 10 inconsideration of the current velocity of the vehicle 10 and the movingdirection of the object. In an embodiment, the pedestrian recognitionapparatus may include a table for storing the distance depending on thesize of the object region in advance. In an embodiment, the pedestrianrecognition apparatus may include a table for storing the movingvelocity of the object depending on a size variation of the objectregion or a positional change of the object region.

In an embodiment, the autonomous emergency braking unit 240 may generatea warning sound when the distance between the vehicle 10 and thecorresponding object is less than a first reference distance, anddecelerate the vehicle 10 and generate a warning sound when the distancebetween the vehicle 10 and the corresponding object is less than asecond reference distance smaller than the first reference distance.

The autonomous emergency braking unit 240 may emergency-brake thevehicle 10 when an object which was not detected in a previous imageframe is detected in the current image frame. For example, theautonomous emergency braking unit 240 may determine that thecorresponding object suddenly appeared around the vehicle 10, therebyemergency-braking the vehicle 10.

The control unit 250 may control the overall operations of thepedestrian recognition apparatus 100, and control data flows andoperations among the image receiving unit 210, the pedestriandetermination unit 220, the pedestrian collision possibilitydetermination unit 230 and the autonomous emergency braking unit 240.

FIG. 8 is a flowchart illustrating the pedestrian recognition processperformed in the pedestrian recognition apparatus of FIG. 1.

Referring to FIG. 8, the image receiving unit 210 receives an image fromthe camera 20 at step S801.

In an embodiment, the image receiving unit 210 may generate an aroundview based on the image acquired from the camera 20. The image receivingunit 210 may generate a time-series around view by combining two or morearound views based on the images taken by the camera 20.

The pedestrian determination unit 220 extracts one or more objects fromthe image received from the camera 20 before the pedestrian candidatedetection process. In an embodiment, the pedestrian determination unit220 may detect edges in the image, and extract one or more objects eachhaving an area equal to or more than a predetermined size distinguishedby the edges. In another embodiment, the pedestrian determination unit220 may extract one or more objects based on a color difference betweenthe background and the object in the image received from the camera 20.

The pedestrian determination unit 220 may detect a pedestrian candidateobject from the one or more objects extracted from the image receivedfrom the camera 20 at step S802.

In an embodiment, the pedestrian determination unit 220 may extract afeature corresponding to a specific feature of a pedestrian (detectiontarget object) from the object through the simple feature analysis, andcompare the extracted feature to detect a pedestrian candidate object.For example, the pedestrian determination unit 220 may detect a verticalcomponent of the one or more objects, and determine a similarity betweenthe vertical component and the pedestrian pattern, in order to detectthe pedestrian candidate object.

The pedestrian determination unit 220 determines whether the pedestriancandidate object is moving, using a difference between a previous imageand the current image at step S803.

The pedestrian determination unit 220 may determine whether thepedestrian candidate object is moving, based on differences of the samepedestrian candidate object in images from the current time to aspecific previous time.

The pedestrian determination unit 220 determines the possibility thatthe moving pedestrian candidate object is a pedestrian, at step S804.

The pedestrian determination unit 220 may perform a HOG operation on themoving pedestrian candidate object, and perform an SVM weight operationon the HOG operation. When the operation result is equal to or more thana preset threshold value, the pedestrian determination unit 220 may setthe pedestrian candidate object to a pedestrian. The pedestrianpossibility determination module 214 may estimate the region of thepedestrian candidate object corresponding to the HOG operation regionbased on the near and far degree of the image received from the camera20.

The pedestrian determination unit 220 may cyclically perform thepedestrian candidate detection process, the mobility determinationprocess and the pedestrian possibility determination process, theredetermining the possibility that one or more objects are pedestrians.

The pedestrian collision possibility determination unit 230 determines acollision possibility of the object set to a pedestrian, based onwhether the object is moving, at step S405.

When the corresponding object is likely to collide with the vehicle, thepedestrian collision possibility determination unit 230 maytransparently overlay a visual guidance on the object, such that theobject with the visual guidance is displayed on the screen.

While various embodiments have been described above, it will beunderstood to those skilled in the art that the embodiments describedare by way of example only. Accordingly, the disclosure described hereinshould not be limited based on the described embodiments.

What is claimed is:
 1. A pedestrian recognition apparatus comprising: acontrol unit including a processor using an algorithm stored on anon-transitory storage device, wherein the algorithm when executed bythe processor causes the control unit to act as: an image receiving unitconfigured to sequentially receive a plurality of images from a camera;and a pedestrian determination unit configured to detect a pedestriancandidate object from one or more objects in the images, and todetermine whether the pedestrian candidate object is moving, using theplurality of images, and to set the pedestrian candidate object to amoving pedestrian candidate object, and to perform a pedestrianpossibility determination process by performing a predefined operationon the moving pedestrian candidate object, and to set the movingpedestrian candidate object to a pedestrian in response to detection ofa value calculated through the predefined operation being equal to ormore than a threshold value, wherein the pedestrian determination unitincludes: an object extraction module configured to extract one or moreobjects from an image received from the image receiving unit, apedestrian candidate detection unit configured to detect a pedestriancandidate object from one or more objects extracted by the objectextraction module, a mobility determination module configured to dividea current image frame and a previous image frame each into blocks of apredetermined size, and in response to detection of the pedestriancandidate object being included in a specific block of the current imageframe, to calculate a sum of absolute difference (SAD) by summing upabsolute values of the pixel value differences between pixels of thespecific block of the current image frame and pixels of correspondingblocks of the previous image frame using an equation below, to determinethe block of the previous image frame having the smallest SAD as acorresponding block of the current image frame, to set the pedestriancandidate object to a moving pedestrian candidate object in response toa determination that the positions of blocks corresponding to each otherin the current image frame and the previous image frame were changed,and a pedestrian possibility determination module configured to performa HOG (Histogram of Oriented Gradient) operation on the movingpedestrian candidate object, to perform a support vector machine (SVM)weight operation on an HOG operation result, and to set the pedestriancandidate object to a pedestrian in response to detection of an SVMweight operation result being equal to or more than a preset thresholdvalue, wherein the SAD is calculated based on the following equation:$\begin{matrix}{{{SAD} = {\sum\limits_{i = 1}^{N}{\sum\limits_{j = 1}^{M}{\left( {{I_{ij}(k)} - {I_{ij}\left( {k - 1} \right)}} \right)}}}},} & \lbrack{Equation}\rbrack\end{matrix}$ wherein I_(ij)(k) represents the pixel value of an i-throw and a j-th column of a block in a k-th image frame, and I_(ij)(k−1)represents the pixel value of an i-th row and a j-th column of a blockin a (k−1)th image frame.
 2. The pedestrian recognition apparatus ofclaim 1, further comprising: a pedestrian collision possibilitydetermination unit configured to overlay a visual guidance on the objectcorresponding to the pedestrian on a display screen of a vehicle inresponse to determination that a distance between the pedestrian and thevehicle is decreasing such that the pedestrian and the vehicle have alikelihood to collide with each other.
 3. The pedestrian recognitionapparatus of claim 1, wherein the pedestrian determination unit isfurther configured to determine the possibility that the one or moreobjects are pedestrians.
 4. The pedestrian recognition apparatus ofclaim 1, wherein the pedestrian determination unit is further configuredto detect vertical components of the one or more objects in thepedestrian candidate object detection process, and to determine asimilarity between the detected vertical components and a pedestrianpattern.
 5. The pedestrian recognition apparatus of claim 1, wherein thepedestrian determination unit is further configured to estimate a regionof the pedestrian candidate object, corresponding to a HOG operationregion, based on a near and far degree of a received image.
 6. Thepedestrian recognition apparatus of claim 5, wherein the pedestriandetermination unit is further configured to adjust a size of the regionof the pedestrian candidate object using a plurality of cells having avariable size.
 7. The pedestrian recognition apparatus of claim 1,further comprising: an autonomous emergency braking unit configured todetermine a collision possibility of the pedestrian with a vehicle, thedetermination is performed based on a distance between the pedestrianand the vehicle, and to control the vehicle by performing at least oneof reducing velocity and generating a warning sound.
 8. Acomputer-implemented pedestrian recognition method comprising:sequentially receiving a plurality of images from a camera; detecting apedestrian candidate object from one or more objects in the images;determining whether the pedestrian candidate object is moving, thedetermination is performed based on the plurality of images; setting thepedestrian candidate object to a moving pedestrian candidate object byperforming a predefined operation on the moving pedestrian candidateobject, and setting the moving pedestrian candidate object to apedestrian in response to detection of a value calculated through thepredefined operation being equal to or more than a threshold value;extracting one or more objects from an image received; detecting apedestrian candidate object from one or more objects extracted; dividinga current image frame and a previous image frame each into blocks of apredetermined size, and in response to detection of the pedestriancandidate object being included in a specific block of the current imageframe; calculating a sum of absolute difference (SAD) by summing upabsolute values of the pixel value differences between pixels of thespecific block of the current image frame and pixels of correspondingblocks of the previous image frame through an equation below;determining the block of the previous image frame having the smallestSAD as a corresponding block of the current image frame; setting thepedestrian candidate object to a moving pedestrian candidate object inresponse to a determination that the positions of blocks correspondingto each other in the current image frame and the previous image framewere changed; and performing a HOG (Histogram of Oriented Gradient)operation on the moving pedestrian candidate object, performing asupport vector machine (SVM) weight operation on an HOG operationresult, and setting the pedestrian candidate object to a pedestrian whenan SVM weight operation result is equal to or more than a presetthreshold value, wherein the SAD is calculated based on the followingequation: $\begin{matrix}{{{SAD} = {\sum\limits_{i = 1}^{N}{\sum\limits_{j = 1}^{M}{\left( {{I_{ij}(k)} - {I_{ij}\left( {k - 1} \right)}} \right)}}}},} & \lbrack{Equation}\rbrack\end{matrix}$ wherein I_(ij)(k) represents the pixel value of an i-throw and a j-th column of a block in a k-th image frame, and I_(ij)(k−1)represents the pixel value of an i-th row and a j-th column of a blockin a (k−1)th image frame.
 9. The pedestrian recognition method of claim8, further comprising: determining that a vehicle and the pedestrianhave a likelihood of colliding with each other in response todetermination that a distance between the pedestrian and the vehicle isdecreasing such that the pedestrian and the vehicle have a likelihood tocollide with each other, and overlaying a visual guidance on the objectcorresponding to the pedestrian on a display screen of the vehicle. 10.The pedestrian recognition method of claim 8, further comprising:determining whether a vehicle and the pedestrian have a likelihood ofcolliding with each other, the determination being performed based on adistance between the pedestrian and the vehicle decreasing, andcontrolling the vehicle by performing at least one of reducing velocityand generating a warning sound.